|
KMID : 1022420230150030075
|
|
Phonetics and Speech Sciences
2023 Volume.15 No. 3 p.75 ~ p.82
|
|
|
Building robust Korean speech recognition model by fine-tuning large pretrained model
|
|
Oh Chang-Han
Kim Cheong-Bin
Park Ki-Young
|
|
|
Abstract
|
|
|
|
Automatic speech recognition (ASR) has been revolutionized with deep learning-based approaches, among which self-supervised learning methods have proven to be particularly effective. In this study, we aim to enhance the performance of OpenAI’s Whisper model, a multilingual ASR system on the Korean language. Whisper was pretrained on a large corpus (around 680,000 hours) of web speech data and has demonstrated strong recognition performance for major languages. However, it faces challenges in recognizing languages such as Korean, which is not major language while training. We address this issue by fine-tuning the Whisper model with an additional dataset comprising about 1,000 hours of Korean speech. We also compare its performance against a Transformer model that was trained from scratch using the same dataset. Our results indicate that fine-tuning the Whisper model significantly improved its Korean speech recognition capabilities in terms of character error rate (CER). Specifically, the performance improved with increasing model size. However, the Whisper model’s performance on English deteriorated post fine-tuning, emphasizing the need for further research to develop robust multilingual models. Our study demonstrates the potential of utilizing a fine-tuned Whisper model for Korean ASR applications. Future work will focus on multilingual recognition and optimization for real-time inference.
|
|
|
KEYWORD
|
|
|
deep learning, machine learning, speech recognition
|
|
|
FullTexts / Linksout information
|
|
|
|
|
|
Listed journal information
|
|
|
|